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ABSTRACT 

The  Common  Warfighting  Symbology  Standard  Bravo  version  (MIL-STD-2525B)  is  a 
contractual  requirement  for  a  number  of  Australian  defence  projects.  To  evaluate  how 
well  MIL-STD-2525B  supports  rapid  and  accurate  detection  of  symbols  in  clutter  the 
current  study  examined  the  speed  and  accuracy  with  which  23  participants  could 
detect  the  presence  of  each  MIL-STD-2525B  air  affiliation  symbol  when  symbol  overlap 
and  icon  presence  were  manipulated.  Detection  efficiency  and  accuracy  were 
significantly  lower  for  MIL-STD-2525B  friend,  assumed  friend,  suspect  and  hostile 
symbols  than  for  neutral  and  unknown  symbols.  Symbol  overlap  significantly  reduced 
detection  efficiency  and  accuracy.  Icon  presence  reduced  detection  efficiency  only.  The 
results  indicate  that  it  was  difficult  to  detect  important  air  symbols  in  MIL-STD-2525B. 
The  study  also  examined  detection  of  friend,  assumed  friend,  suspect  and  hostile 
symbols  from  the  updated  version  of  the  standard,  MIL-STD-2525C.  The  MIL-STD- 
2525C  representation  of  suspect  affiliation  improved  detection  efficiency  for  suspect 
and  hostile  symbols  compared  to  MIL-STD-2525B  but  the  representation  of  assumed 
friend  affiliation  produced  poorer  detection  efficiency  for  assumed  friend  and  friend 
symbols.  Thus  the  change  to  the  suspect  symbol  proposed  in  MIL-STD-2525C 
appeared  to  be  effective,  but  further  work  is  required  to  modify  the  assumed  friend 
symbol  within  the  scope  of  the  standard  to  improve  the  detection  performance  of 
friend  and  assumed  friend  symbols. 
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Detection  of  Common  Warfighting  Symbology 
(MIL-STD-2525)  Air  Symbols 

Executive  Summary 

Military  tactical  command  and  control  displays  typically  use  symbols  to  convey 
information  about  the  location,  affiliation,  and  identity  of  objects  within  the  battle 
space.  These  symbols  must  be  detected  and  understood  quickly  and  accurately  if  they 
are  to  effectively  support  situation  awareness  and  decision  making. 

Current  defence  projects,  e.g.  JP2089  Phase  2A,  will  introduce  the  Common  Warfighting 
Symbology  standard  into  naval  and  joint  tactical  displays  and  currently  mandate  the  use 
of  MIL-STD-2525B.  The  proposed  implementation  of  MIL-STD-2525B  may  make  it 
difficult  to  distinguish  hostile  symbols  from  suspect  symbols  and  friend  symbols  from 
assumed  friend  symbols,  particularly  when  symbols  are  overlapping  and  when  symbol 
icons  are  displayed.  A  more  recent  version  of  the  standard,  MIL-STD-2525C,  has  been 
released  in  which  changes  have  been  made  to  the  suspect  and  assumed  friend  symbols. 
It  may  be  that  MIL-STD-2525C  will  allow  these  symbols  to  be  more  easily 
distinguished. 

The  current  study  aimed  to: 

1.  Measure  how  effectively  each  MIL-STD-2525B  air  affiliation  symbol  could  be 
detected  by  operators  and  how  detection  is  affected  by  symbol  overlap  and  the 
presence  of  icons  within  the  symbol  frames. 

2.  Evaluate  the  effect  of  the  changes  to  assumed  friend  and  suspect  affiliation 
symbols  contained  in  MIL-STD-2525C. 

Fast  and  accurate  detection  of  symbol  affiliation  is  a  precursor  to  effective  tactical 
picture  comprehension  and  decision  making  and  the  current  study  used  a  visual 
search  task  to  explore  the  speed  and  accuracy  with  which  each  affiliation  could  be 
detected  when  displayed  among  a  set  of  all  other  affiliations. 

The  current  study  found  that: 

•  MIL-STD-2525B  assumed  friend,  friend  suspect  and  hostile  symbols  do  not  support 
efficient  and  accurate  detection. 

•  Symbol  overlap  caused  a  substantial  reduction  in  detection  efficiency  and 
accuracy,  and  the  presence  of  icons  caused  a  smaller  reduction  in  detection 
efficiency  only.  The  presence  of  icons  did  not  increase  the  effect  of  overlap  but 
this  may  have  been  due  to  the  specific  overlap  geometry  used  in  the  study. 

•  The  MIL-STD-2525C  representation  of  suspect  improved  detection  performance 
for  suspect  and  hostile  affiliations  compared  to  MIL-STD-2525B. 

•  However,  the  MIL-STD-2525C  representation  of  assumed  friend  worsened 
detection  performance  for  assumed  friend  and  friend  affiliations  compared  to 
MIL-STD-2525B. 
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Several  recommendations  arise  from  the  current  study: 

•  Changes  should  be  made  to  how  the  Common  Warfighting  Symbology  standard  is 
implemented  in  RAN  projects.  At  a  minimum,  changes  are  needed  to  suspect 
and  assumed  friend  symbols.  The  changes  could  either  involve  continuing  to 
use  MIL-STD-2525B  but  changing  the  colour  of  suspect  and  assumed  friend 
symbols  or  moving  to  MIL-STD-2525C. 

•  In  either  case  further  work  is  required  to  develop  and  empirically  evaluate  new 
assumed  friend  and  suspect  symbols  that  are  consistent  with  the  chosen  version 
of  the  Common  Warfighting  Symbology  standard  and  will  allow  these  symbols  to 
be  easily  distinguished  from  other  symbols  within  the  symbols  set.  Other 
relevant  symbol  recommendations  proposed  in  MIL-STD-2525C,  such  as  the 
use  of  purple  to  denote  civilian  air  tracks,  should  also  be  investigated. 

•  Conduct  a  follow-on  study  to  clarify  the  effect  of  icon  presence  on  symbol 
detection  under  different  overlap  conditions. 

•  In  addition  to  detection  performance,  examine  the  ability  of  the  symbols  and 
icons  to  support  the  higher-level  tasks  of  symbol  classification  and  tactical 
picture  comprehension. 

•  Identify  and  experimentally  evaluate  display  options  that  eliminate  symbol 
overlap  but  still  allow  accurate  picture  comprehension  for  use  in  future  tactical 
displays.  However,  given  that  symbol  overlap  is  likely  to  be  an  ongoing 
problem  with  current  and  future  displays  the  effects  of  symbol  overlap  should 
continue  to  be  explored  in  future  studies. 
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1.  Introduction 


Military  tactical  command  and  control  displays  typically  use  symbols  to  convey 
information  about  the  location,  affiliation,  and  identity  of  objects  within  the  battle  space. 
These  symbols  must  be  detected  and  understood  quickly  and  accurately  if  they  are  to 
effectively  support  situation  awareness  and  decision  making.  A  range  of  symbology  sets 
exist,  including  the  Naval  Combat  Data  System  (NCDS)  which  is  currently  used  on  some 
Royal  Australian  Navy  platforms,  the  North  Atlantic  Treaty  Organisation  Standardisation 
Agreement  (NATO  STANAG  4420)  and  the  Common  Warfighting  Symbology  standard 
(MIL-STD-2525).  Current  Australian  defence  projects,  e.g.  JP2089  and  SEA4000,  will 
introduce  the  Common  Warfighting  Symbology  standard  into  naval  and  joint  tactical  displays 
and  this  symbol  set  is  the  focus  of  the  current  study. 

Symbols  in  the  Common  Warfighting  Symbology  set  are  composed  of  a  frame  which  uses 
shape  and  colour  to  denote  the  category  (space,  air,  surface  or  subsurface)  and  the 
affiliation  (friend ,  assumed  friend,  neutral,  suspect,  hostile  or  unknown )  of  an  object.  Frames 
may  be  filled,  in  which  case  the  interior  of  the  frame  is  filled  with  the  affiliation  colour,  or 
unfilled,  when  the  interior  of  the  frame  is  transparent  but  the  frame  border  is  the  colour  of 
the  affiliation.  Symbols  may  also  contain  an  icon  or  letter  located  inside  the  frame  that 
represents  the  platform  type  of  the  object  (e.g.  fighter,  helicopter,  missile)  and  use 
modifiers  to  provide  additional  information  about  an  object  such  as  leader  lines  that 
denote  course  and  speed  and  text  modifiers  to  denote  track  number,  call  sign, 
altitude/  depth,  IFF  mode  and  engagement  status. 

The  Common  Warfighting  Symbology  standard  recognises  that  different  operational 
requirements  may  require  that  different  amounts  of  information  about  an  object  be 
displayed.  Accordingly  it  provides  the  flexibility  to  allow  for  frames,  fill  and  icons  to  be 
independently  turned  on  or  off  (DoD,  2007,  p.  62).  The  standard  also  recommends  default 
colours  for  each  affiliation  but  advises  that  "implementors  should  include  sufficient 
usability  testing  to  ensure  effective  operator  performance  when  selecting  colours  to  render 
the  symbology"  (DoD,  2007,  p.  71). 

The  Common  Warfighting  Symbology  has  also  undergone  a  number  of  revisions  since  its 
initial  release  in  September  1994.  The  most  recent  update  has  been  from  MIL-STD-2525B 
Change  2  to  MIL-STD-2525C  in  November  2008  (DoD,  2008)  which,  among  other  things, 
introduced  changes  to  how  the  assumed  friend  and  suspect  affiliations  are  displayed. 
MIL-STD-2525B  uses  a  small  question  mark  near  the  upper-right  corner  of  the  symbol 
frame  to  denote  these  affiliations  whereas  MIL-STD-2525C  uses  a  dashed  white  and 
coloured  frame.  These  changes  were  a  result  of  empirical  studies  conducted  by  the  Naval 
Surface  Warfare  Centre  (Davidson  &  Wetzel,  2007)  and,  although  details  of  the  studies 
were  not  published,  suggest  that  usability  problems  may  exist  with  MIL-STD-2525B.  One 
of  these  problems  may  be  that  the  use  of  a  question  mark  made  assumed  friend  and  suspect 
affiliations  difficult  to  detect  on  cluttered  displays.  Common  Warfighting  Symbology  also 
allows  for  icons  to  be  displayed  with  the  symbol  frames.  While  this  may  provide 
classification  information  about  tracks,  the  presence  of  icons  within  the  symbol  frames  has 
been  shown  to  increase  detection  time  (McFadden,  Jeon,  Li,  &  Minniti,  2008)  and  the 
presence  of  icons  may  exacerbate  the  difficulty  of  detecting  MIL-STD-2525B  suspect  and 
assumed  friend  symbols  by  obscuring  the  question  marks. 
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Current  defence  projects  mandate  the  use  of  MIL-STD-2525B  and  require  that  icons  be 
displayed.  Given  the  possible  problems  identified  above,  the  current  study  had  two 
objectives: 

1.  To  evaluate  the  detection  of  MIL-STD-2525B  air  symbols  and  how  detection  is 
affected  by  symbol  overlap  and  icon  presence. 

2.  To  evaluate  the  effect  of  the  changes  to  assumed  friend  and  suspect  symbols 
contained  in  MIL-STD-2525C. 

The  current  study  used  a  visual  search  task  to  identify  the  speed  and  accuracy  with  which 
affiliation  symbols  could  be  detected  when  displayed  among  other  affiliation  symbols 
while  symbol  overlap  and  icon  presence  were  manipulated.  In  support  of  objective  1,  the 
current  study  included  each  MIL-STD-2525B  air  affiliation  symbol  (friend ,  assumed  friend, 
neutral,  suspect,  hostile,  unknown).  To  support  objective  2,  detection  of  assumed  friend, 
suspect,  friend  and  hostile  affiliations  from  the  MIL-STD-2525C  symbol  set  was  tested. 
Assumed  friend  and  suspect  affiliations  were  tested  as  changes  have  been  made  to  these 
symbols  in  MIL-STD-2525C.  Friend  and  hostile  were  also  tested  as,  although  these  symbols 
did  not  change,  their  detection  may  be  affected  by  the  changes  to  the  assumed  friend  and 
suspect  symbols.  The  study  used  only  air  symbols  in  order  to  control  the  number  of 
symbols  that  needed  to  be  tested  and  also  because  detection  of  air  and  missile  threats  is 
particularly  time-critical. 

Fast  and  accurate  symbol  detection  must  occur  prior  to  tactical  picture  comprehension, 
and  measures  of  single-symbol  detection  performance  under  simple  display  conditions 
provide  a  controlled  means  of  identifying  symbols  which  may  hinder  picture 
comprehension  and  therefore  decision  making  in  the  more  complicated  displays  that  are 
likely  to  be  encountered  in  real  military  operations. 


2.  Method 


2.1  Participants 

Twenty-three  employees  of  the  Defence  Science  and  Technology  Organisation  (DSTO), 
Australia,  participated  in  the  study.  The  mean  age  was  38.1  years  (SD  =  8.0  years).  All  had 
normal  colour  vision  tested  using  the  12-plate  Ishihara  colour  test.  Participants  wore  their 
normal  corrective  lenses  (glasses,  contact  lenses)  as  required. 

2.2  Materials  and  Apparatus 

The  experiment  was  controlled  using  PsyToolkit  (Stoet,  2010)  and  run  on  a  computer 
using  Ubuntu  10.10  with  a  P4  3.6  GHz  processor,  1  Gb  of  RAM  and  a  NVIDIA  GeForce 
7800  GTX  256  Mb  dual-link  DVI  video  card  to  drive  a  Dell  30"  monitor  at  its  native 
resolution  of  2560  x  1600  pixels  at  60  Hz. 

Each  trial  presented  a  visual  stimulus  that  displayed  a  number  of  symbols  located  within  a 
circle  of  radius  of  600  pixels  (151  mm)  centred  in  the  monitor.  The  circle  subtended  a 
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visual  angle  of  26  degrees  at  the  approximate  viewing  distance  of  650  mm.  The  symbols 
were  randomly  distributed  on  a  uniform  regular  grid  within  the  circle  with  a  minimum 
spacing  of  98  pixels. 


The  symbol  set  used  is  shown  in  Figure  1.  The  frame  of  each  symbol  had  height  of 
between  23  and  35  pixels  and  a  width  of  between  21  and  27  pixels,  subtending  a  visual 
angle  of  approximately  30  minutes  of  arc  at  the  approximate  viewing  distance  of  650  mm. 
The  R:G:B  values  were  255:48:49  for  hostile  and  suspect,  0:255:255  for  friend  and  assumed 
friend,  0:255:0  for  neutral  and  255:255:0  for  unknown.  The  white  in  the  2525C  assumed  friend 
and  suspect  frames  was  255:255:255. 


Affiliation 


No  Icon 


With  Icon 


Friend 

Assumed  friend  -  2525B 

Assumed  friend  -  2525C 

Neutral 

Suspect  -  2525B 
Suspect  -  2525C 

Hostile 

Unknown 


R  P  (f!  fp) 


R  Rl  R  R  R 


<9)  <Q>  (?)  <R>  <9) 


Figure  1  The  symbols  used  in  the  current  study 


2.3  Measures 

Response  time  and  accuracy  were  recorded  for  each  trial.  Response  time  was  the  period 
between  stimulus  display  and  when  a  participant  indicated,  via  a  key  press,  whether  a 
symbol  of  a  particular  affiliation  was  present  or  not.  Accuracy  was  whether  the  decision 
was  correct  or  not. 

2.4  Design 

The  study  tested  the  detection  performance  of  each  affiliation  symbol  in  MIL-STD-2525B 
(friend,  assumed  friend,  neutral,  suspect,  hostile  and  unknown)  and  a  subset  of  affiliation 
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symbols  from  MIL-STD-2525C  (friend,  assumed  friend,  suspect  and  hostile).  In  each  case  the 
target  symbols  were  displayed  among  all  other  affiliation  symbols  within  their  symbol  set. 
Symbol  overlap  and  icon  presence  were  manipulated.  Symbols  were  either  not 
overlapping  or  two  symbols  overlapped  by  approximately  50%  along  an  axis  rotated 
45  degrees  from  horizontal  as  shown  in  Figure  2.  Symbols  were  displayed  either  with  or 
without  icons  as  shown  in  Figure  1.  Either  10  or  40  symbols  were  displayed  which  allowed 
the  response  time  slope  to  be  calculated  as  described  in  2.6.  The  target  symbol  was  present 
for  50%  of  trials.  Each  condition  was  fully  crossed  and  five  trials  were  presented  in  each 
condition  which  required  10  (affiliation)  x  2  (10  or  40  symbols)  x  2  (icon  or  no  icon)  x 
2  (overlap  or  no  overlap)  x  2  (target  present  or  not  present)  x  5  =  800  trials.  Trials  were 
grouped  in  blocks  of  symbol  affiliation  with  block  order  determined  by  a  balanced  Latin- 
square  design.  The  presentation  order  of  trials  within  blocks  was  randomized.  Each 
participant  completed  all  conditions. 


(L? 

K>) 

Ft 

[FI  <P) 

<KV 

A'  (9) 

& 

ft 

Non-Overlapping  Symbols  Overlapping  Symbols 

Figure  2  Example  of  the  non-overlap  (left  panel)  and  overlap  conditions  for  symbols  with  icons 

2.5  Procedure 

Participants  were  seated  in  front  of  the  computer  monitor,  briefed  on  the  experimental 
task  and  undertook  a  short  gaze-direction  calibration  procedure  to  allow  the  collection  of 
eye-tracking  data  not  reported  here.  Participants  then  completed  a  block  of  16  practice 
trials,  one  of  each  condition,  using  a  neutral  affiliation  target.  Participants  then  completed  a 
seven-item  mood  scale,  not  reported  here,  before  commencing  the  first  of  the  10  trial 
blocks.  Each  block  took  approximately  5  minutes  to  complete  and  participants  were  given 
the  opportunity  to  break  between  blocks.  Immediately  after  the  completion  of  the  final 
block  participants  again  completed  the  seven-item  mood  scale  and  the  colour  vision  test. 
The  entire  experimental  session  lasted  for  approximately  70  minutes. 

2.6  Analysis 

Response  time  and  accuracy  will  be  analysed  separately.  The  average  response  time  of 
trials  with  a  correct  response  was  calculated  for  each  condition.  From  this  the  response 
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time  slope  (RT  slope)  was  calculated  as  the  difference  in  response  time  between  40-symbol 
trials  and  10-symbol  trials  in  each  condition  divided  by  the  difference  in  the  number  of 
symbols  in  each  trial.  RT  slope  provides  a  measure  of  the  efficiency  of  the  visual  search 
independent  of  set  size  which  can  be  compared  to  other  studies  (Wolfe,  1998).  The 
percentage  of  trials  in  each  condition  with  an  incorrect  response  was  used  as  the  measure 
of  accuracy.  Repeated-measures  analysis  of  variance  (ANOVA)  was  used  for  the  analyses 
with  Greenhouse-Geisser  corrections  applied  when  violations  of  the  assumption  of 
sphericity  were  identified. 


3.  Results 


The  effects  of  overlap  and  icon  presence  on  detection  performance  for  MIL-STD-2525B 
symbols  will  be  presented  initially.  This  analysis  will  combine  target-present  and  target- 
absent  trials  to  explore  how  these  factors  influence  the  ability  to  detect  or  reject  the 
presence  of  particular  symbols.  Next  a  comparison  of  the  detection  performance  of  each 
MIL-STD-2525B  affiliation  will  be  presented.  This  analysis  will  use  only  target-present 
trials  as  the  focus  is  on  how  effectively  each  symbol  can  be  detected  and  will  allow  the  RT 
slope  results  to  be  compared  with  other  studies.  Finally  a  comparison  of  the  detection 
performance  of  the  MIL-STD-2525B  and  MIL-STD-2525C  representations  oi  friend,  assumed 
friend,  suspect  and  hostile  symbols  will  be  presented.  This  analysis  will  also  use  only  target- 
present  trials. 

3.1  Effect  of  Symbol  Overlap  and  Icon  Presence  in  MIL-STD-2525B 

Plots  of  the  RT  slope  and  error  for  the  symbol  overlap  and  icon  presence  manipulations  for 
the  MIL-STD-2525B  symbols  are  shown  in  Figure  3.  Overlapping  symbols  had  a 
significantly  higher  RT  slope  and  error  rate  than  non-overlapping  symbols, 
F(l,22)  =  168.23,  p  <  .001,  tp,2  =  .884  and  F(l,22)  =  159.94,  p  <  .001,  tp,2  =  .879  respectively. 
Thus  searching  overlapping  symbols  was  less  efficient  and  more  error  prone  than 
searching  symbols  that  did  not  overlap.  The  presence  of  icons  significantly  increased  RT 
slope,  F(l,22)  =  11.31,  p  =  .003,  t]p2  =  .340,  but  not  errors,  F(l,22)  =  .02,  p  =  .889,  rp,2  =  .001. 
Thus  searching  for  symbols  with  icons  was  less  efficient  than  searching  for  symbols 
without  icons  but  this  did  not  translate  into  increased  errors.  The  overlap  x  icon 
interaction  was  not  significant  for  RT  slope  or  error,  F(l,22)  =  2.08,  p  =  .164,  tjp2  =  .086  and 
F(l,22)  =  .03,  p  =  .866,  rp,2  =  .001.  This  indicates  that  the  effect  of  symbol  overlap  was  not 
moderated  by  the  presence  of  icons. 
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Figure  3  Mean  RT  slope  (left  plot)  and  errors  (right  plot)  for  the  overlap  and  icon  manipulations 
for  MIL-STD-2525B  symbols.  Error  bars  represent  95th  percentile  confidence  intervals. 


3.2  Comparison  of  MIL-STD-2525B  Symbol  Detection  Performance 

The  next  step  was  to  compare  the  detection  performances  of  each  MIL-STD-2525B  symbol 
using  only  target-present  trials  and  plots  of  average  RT  slope  and  error  for  overlapping 
and  non-overlapping  symbols  are  shown  in  Figure  4.  Icon  presence  was  collapsed  across 
trials  due  to  its  small  effect  size.  There  were  significant  differences  in  RT  slope  and  error 
between  symbols,  F(5,110)  =  32.39,  p  <  .001,  rjp2  =  .595  and  F(5,110)  =  9.31  p  <  .001,  r/P2  =  .297 
respectively.  Pairwise  comparisons  indicated  that  unknown,  neutral  and  assumed  friend  did 
not  differ  in  RT  slope  but  had  significantly  lower  RT  slopes  than  friend,  suspect  and  hostile 
which  did  not  significantly  differ  from  each  other.  Unknown  and  neutral  symbols  had  RT 
slopes  of  approximately  12  ms  /  symbol  whereas  the  other  symbols  had  an  average  RT 
slope  of  approximately  25  ms  /  symbol.  A  similar  pattern  was  obtained  for  error,  with  the 
exception  that  assumed  friend  grouped  with  friend,  suspect  and  hostile  affiliations.  There  was 
also  a  significant  overlap  x  ID  interaction  for  both  RT  slope  and  error,  F(3.05,67.03)  =  8.86, 
p  <  .001,  pp2  =  .287  and  F(5,110)  =  14.64  p  <  .001,  rjP2  =  .400  respectively.  Overlap  decreased 
efficiency  and  increased  errors  more  for  assumed  friend,  friend,  suspect  and  hostile  symbols 
than  for  unknown  and  neutral  symbols. 
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Figure  4  Mean  RT  Slope  and  Error  for  each  MIL-STD-2525B  affiliation  symbol  for  target- 
present  trials.  Error  bars  represent  the  95th  percentile  confidence  intervals.  The  dotted 
line  separates  the  two  post-hoc  test  groups. 

3.3  Comparison  between  MIL-STD-2525B  and  MIL-STD-2525C  Friend , 
Assumed  friend,  Suspect  and  Hostile  Symbols 

MIL-STD-2525C  introduces  changes  to  assumed  friend  and  suspect  symbols  which  may 
influence  their  detection  performance  and  also  that  of  friend  and  hostile  symbols  which 
they  closely  resemble.  The  RT  slope  and  error  results  for  assumed  friend  and  friend  symbols 
in  MIL-STD-2525B  and  MIL-STD-2525C  are  shown  in  Figure  5.  Overall,  the  MIL-STD- 
2525C  representation  of  assumed  friend  led  to  worse  detection  performance  for  both 
assumed  friend  and  friend  symbols  than  the  MIL-STD-2525B  representation,  with  a  higher 
RT  slope  and  greater  errors,  F(l,22)  =  102.9,  p  <  .001,  rjp2  =  .824  and  F(l,22)  =  62.69,  p  <  .001, 
r]p2  =  .740  respectively.  Detection  of  assumed  friend  symbols  was  more  efficient  than  friend 
symbols,  F(l,22)  =  15.29,  p  =  .001,  rjp2  =  .410,  and  although  this  gap  appeared  to  reduce  in 
MIL-STD-2525C  the  symbol  x  symbol  set  interaction  just  failed  to  reach  significance  for  RT 
slope,  F(l,22)  =  3.96,  p  =  .059,  rjv2  =  .152.  The  symbol  x  symbol  set  interaction  was 
significant  for  error,  F(l,22)  =  24.85,  p  <  .001,  rjP2  =  .530,  with  assumed  friend  having  a  higher 
miss  rate  than  friend  in  MIL-STD-2525C  whereas  they  had  similar  miss  rates  in 
MIL-STD-2525B. 
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Figure  5  Mean  RT  Slope  (left  plot)  and  error  (right  plot)  for  the  MIL-STD-2525B  and 
MIL-STD-2525C  representations  of  assumed  friend  and  friend  affiliations.  Error  bars 
represent  the  95th  percentile  confidence  intervals. 


The  RT  slope  and  error  results  for  suspect  and  hostile  symbols  in  MIL-STD-2525B  and 
MIL-STD-2525C  are  shown  in  Figure  6.  Unlike  assumed  friend,  the  change  to  the  suspect 
symbol  in  MIL-STD-2525C  improved  search  performance  for  both  suspect  and  hostile 
symbols.  Overall  both  their  RT  slope  and  error  were  lower  for  MIL-STD-2525C  than 
MIL-STD-2525B,  F(l,22)  =  96.85,  p  <  .001,  pf  =  .815  and  F(l,22)  =  5.73,  p  =  .026,  pf  =  .207 
respectively.  There  was  a  significant  set  x  symbol  interaction  for  both  RT  slope  and  error, 
F(l,22)  =  23.13,  p  <  .001,  pP2  =  .512  and  F(l,22)  =  6.89,  p  =  .015,  pf  =  .239  respectively. 
Suspect  symbols  were  detected  more  efficiently  than  hostile  symbols  in  MIL-STD-2525B,  but 
this  was  reversed  in  MIL-STD-2525C  where  hostile  symbols  were  detected  more  efficiently 
than  suspect  symbols.  The  miss  rate  for  hostile  symbols  was  also  decreased  in 
MIL-STD-2525C  without  changing  the  miss  rate  for  suspect  symbols. 
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Figure  6  Mean  RT  Slope  (left  plot)  and  error  (right  plot)  for  the  MIL-STD-2525B  and 
MIL-STD-2525C  representations  of  suspect  and  hostile  affiliations.  Error  bars 
represent  the  95th  percentile  confidence  intervals. 
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4.  Discussion 


The  current  study  investigated  how  symbol  overlap  and  icon  presence  affected  symbol 
detection  and  performance  and  how  quickly  and  accurately  each  MIL-STD-2525B 
affiliation  symbol  could  be  detected  when  displayed  among  all  other  affiliation  symbols  in 
the  set.  The  study  also  examined  how  the  changes  to  assumed  friend  and  suspect  symbols 
proposed  in  MIL-STD-2525C  affected  detection  performance  of  these  symbols  and  any 
effect  that  these  changes  may  have  had  on  friend  and  hostile  symbols  which  are  similar  in 
appearance. 

4.1  Effect  of  Symbol  Overlap  and  Icon  Presence  in  MIL-STD-2525B 

Symbol  overlap  substantially  reduced  search  efficiency  and  increased  errors.  It  was  easier 
to  detect  a  symbol  when  it  is  displayed  in  isolation  than  when  it  was  overlapping  another 
symbol  which  created  a  more  complicated  combined  figure  from  which  its  shape  must  be 
identified.  In  this  regard  symbol  overlap  might  be  considered  similar  to  a  non-uniform 
background,  and  other  studies  have  found  background  characteristics  to  have  a 
substantial  effect  on  the  detection  of  even  simple,  familiar  targets  (Yamani  &  McCarley, 
2011). 

As  suspected,  icon  presence  led  to  a  less  efficient  search.  However,  the  size  of  the  effect 
was  only  moderate  and  did  not  lead  to  any  change  in  error  rates.  While  the  presence  of 
icons  may  provide  amplifying  information  about  objects  that  may  assist  tactical  picture 
comprehension,  if  they  are  not  needed  they  should  be  turned  off.  Hence  it  is 
recommended  that  icon  presence  be  operator  selectable. 

While  there  was  a  tendency  for  search  efficiency  for  overlapping  symbols  to  be  less 
efficient  when  icons  were  present  than  when  they  weren't,  this  effect  did  not  reach 
significance.  This  was  contrary  to  expectations  that  the  presence  of  icons  would  obstruct 
the  shape  and  colour  of  the  overlapping  frames  and  make  targets  more  difficult  to 
identify,  especially  for  the  MIL-STD-2525B  assumed  friend  and  suspect  symbols.  However,  it 
may  be  that  the  specific  overlap  geometry  used  in  the  current  study  masked  a  potentially 
larger  effect  of  icons.  An  inspection  of  Figure  2  indicates  that  in  the  overlapping  symbol 
pairs  the  question  mark  of  one  symbol  overlaps  with  the  frame,  not  the  icon,  of  the  other 
symbol.  This  meant  that  the  question  mark  was  equally  obscured  in  the  icon  and  no  icon 
conditions  and  may  explain  why  the  effect  of  overlap  was  independent  of  icon  presence.  It 
may  be  that  a  different  overlap  geometry  in  which  the  question  marks  coincided  with  the 
icon  and  not  the  frame  of  the  other  symbol  may  reveal  a  more  substantial  effect  of  icons. 

4.2  Detection  of  MIL-STD-2525B  Symbols 

The  results  of  the  current  study  clearly  indicate  that  substantial  differences  exist  between 
the  detection  performance  of  individual  MIL-STD-2525B  symbols.  Unknown  and  neutral 
symbols  were  detected  more  efficiently  and  with  fewer  misses  than  friend,  assumed  friend, 
suspect  and  hostile  symbols.  The  target-present  RT  slopes  for  unknown  and  neutral  symbols 
were  approximately  12  ms  /  symbol  which  is  considered  to  be  an  efficient  search  (Wolfe, 
1998)  and  indicates  that  these  symbols  tend  to  "pop  out"  of  the  display.  In  contrast  the 
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average  target-present  RT  slopes  for  the  other  symbols  were  around  25  ms  /  symbol 
which  indicates  that  inefficient  searches  were  required  to  detect  these  symbols. 

The  difficulties  observed  with  MIL-STD-2525B  friend,  assumed  friend,  suspect  and  hostile 
symbols  are  particularly  evident  in  the  overlapping  condition  where  their  detection 
performance  degrades  substantially  more  than  the  unknown  and  neutral  symbols  and 
where  their  presence  was  missed  on  an  average  of  approximately  15%  of  trials.  This  would 
seem  to  be  a  worryingly  high  error  rate. 

Another  possible  problem  with  the  MIL-STD-2525B  symbol  set  is  that  friend  symbols  are 
detected  with  less  efficiency  than  assumed  friend  symbols  and  that  hostile  symbols  are 
detected  with  less  efficiency  than  suspect  symbols.  It  would  seem  desirable  for  this 
situation  to  be  reversed,  as  it  would  appear  important  to  be  able  to  quickly  identify  the 
location  of  friendly  assets  among  a  background  of  civilian  or  other  assumed  friend  tracks 
and  to  be  able  to  distinguish  hostile  tracks  that  can  be  engaged  from  tracks  that  are  suspect 
but  cannot  be  engaged.  These  relative  detection  efficiencies  are  likely  to  be  caused  by  the 
use  of  a  question  mark  to  denote  assumed  friend  and  suspect  affiliations.  The  presence  of  a 
feature  is  easier  to  find  than  its  absence  (Treisman  &  Gormican,  1988;  Treisman  &  Souther, 
1985),  and  it  appears  more  difficult  to  detect  the  symbol  that  does  not  have  a  question 
mark  than  one  that  does. 

Thus  problems  exist  with  the  MIL-STD-2525B  representation  of  friend,  assumed  friend, 
suspect  and  hostile  symbols  which  appear  to  require  the  redesign  of  the  assumed  friend  and 
suspect  symbols. 

4.3  Evaluation  of  Symbol  Changes  in  MIL-STD-2525C 

It  may  be  that  the  problems  observed  in  the  detection  performance  of  the  MIL-STD-2525B 
friend,  assumed  friend,  suspect  and  hostile  symbols  can  be  overcome  by  adopting  the  changes 
proposed  in  MIL-STD-2525C  that  assumed  friend  and  suspect  symbols  use  a  white  and 
coloured  dashed  frame  line.  However,  the  results  of  the  current  study  present  a  mixed 
assessment  of  this  approach.  The  MIL-STD-2525C  representation  of  suspect  led  to  a 
substantial  improvement  in  detection  performance  for  both  suspect  and  hostile  symbols 
compared  to  the  MIL-STD-2525B  representation.  Search  efficiency  for  the  MIL-STD-2525C 
suspect  and  hostile  symbols  reduced  to  approximately  15  ms  /  symbol  and  10  ms  /  symbol 
respectively,  which  is  near  the  results  obtained  for  unknown  and  neutral  symbols.  The 
MIL-STD-2525C  representation  also  resulted  in  hostile  symbols  having  higher  search 
efficiency  than  suspect  symbols. 

However,  the  MIL-STD-2525C  representation  of  assumed  friend  used  in  the  current  study 
led  to  worse  search  efficiency  and  error  performance  for  assumed  friend  and  friend  symbols 
than  the  MIL-STD-2525B  representation.  Participants  found  it  difficult  to  distinguish  the 
dashed  cyan  and  white  frame  of  the  assumed  friend  symbol  from  the  solid  cyan  frame  of  the 
friend  symbol  and  commented  that  there  was  not  sufficient  contrast  between  the  cyan  and 
white  colours.  However,  the  observed  effect  might  have  been  caused  by  any  combination 
of  colour  contrast,  dash  length,  frame  shape,  line  width  or  other  frame  characteristics.  The 
specific  monitor  used  may  also  have  contributed  to  the  problem.  What  is  obvious  is  that  a 
different  implementation  of  assumed  friend  is  required  than  was  used  in  the  current  study. 
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4.4  Limitations  of  Current  Study 

A  number  of  limitations  associated  with  the  current  study  need  to  be  noted.  First,  the 
experiment  used  a  single-target  detection  task.  This  obviously  differs  from  how  tactical 
displays  are  used  in  operational  settings,  which  is  to  generate  a  comprehensive 
understanding  of  the  battle  environment  and  support  effective  decision  making.  Flowever, 
fast  and  accurate  detection  and  recognition  of  symbols  is  a  necessary  first  step  to  this 
higher  goal  and  problems  with  symbol  detection  will  interfere  with  this  process.  It  may  be 
that  the  reduction  in  detection  efficiency  observed  when  icons  are  present  is  offset  by  an 
increase  in  situation  awareness  made  possible  by  the  additional  information  that  icons  can 
provide  about  battle-space  entities.  Future  studies  should  evaluate  whether  this  is  the  case. 

A  second  potential  limitation  is  that  the  current  study  used  a  display  that  is  likely  to  be 
simpler  than  would  be  experienced  in  an  operational  environment.  Relatively  few  symbols 
were  present  which  had  no  leader  lines  or  modifiers  such  as  track  number  or  engagement 
status.  The  symbols  were  also  presented  on  a  uniform  black  background  with  none  of  the 
background  clutter  that  may  be  present  on  operational  tactical  displays  due  to  background 
mapping  or  radar  return  video.  However,  these  limitations  are  likely  to  make  the  results  of 
the  current  study  conservative,  almost  a  "best  case"  outcome,  and  it  may  be  that  the  effects 
observed  here  will  be  magnified  in  operational  environments. 

A  third  limitation  is  that  the  current  study  used  air  symbols  only.  Air  tracks  are  fast 
moving  and  must  be  identified  and  responded  to  quickly,  however,  air,  surface  and  sub¬ 
surface  symbols  will  be  simultaneously  present  on  many  naval  and  joint  tactical  displays 
and  future  studies  should  include  symbols  from  all  battle  dimensions. 


5.  Conclusions  and  Recommendations 


The  current  study  found  that: 

MIL-STD-2525B  assumed  friend,  friend  suspect  and  hostile  symbols  do  not  support  efficient 
and  accurate  detection. 

Symbol  overlap  caused  a  substantial  reduction  in  detection  efficiency  and  accuracy,  and 
the  presence  of  icons  caused  a  smaller  reduction  in  detection  efficiency  only.  The  presence 
of  icons  did  not  increase  the  effect  of  overlap  but  this  may  have  been  due  to  the  specific 
overlap  geometry  used  in  the  study. 

The  MIL-STD-2525C  representation  of  suspect  improved  detection  performance  for  suspect 
and  hostile  affiliations  compared  to  MIL-STD-2525B. 

However,  the  MIL-STD-2525C  representation  of  assumed  friend  worsened  detection 
performance  for  assumed  friend  and  friend  affiliations  compared  to  MIL-STD-2525B. 

Several  recommendations  arise  from  the  current  study: 
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Changes  should  be  made  to  how  the  Common  Warfighting  Symbology  standard  is 
implemented  in  RAN  projects.  At  a  minimum,  changes  are  needed  to  suspect  and 
assumed  friend  symbols.  The  changes  could  either  involve  continuing  to  use 
MIL-STD-2525B  but  changing  the  colour  of  suspect  and  assumed  friend  symbols  or 
moving  to  MIL-STD-2525C. 

In  either  case  further  work  is  required  to  develop  and  empirically  evaluate  new  assumed 
friend  and  suspect  symbols  that  are  consistent  with  the  chosen  version  of  the 
Common  Warfighting  Symbology  standard  and  will  allow  these  symbols  to  be  easily 
distinguished  from  other  symbols  within  the  symbols  set.  Other  relevant  symbol 
recommendations  proposed  in  MIL-STD-2525C,  such  as  the  use  of  purple  to  denote 
civilian  air  tracks,  should  also  be  investigated. 

Conduct  a  follow-on  study  to  clarify  the  effect  of  icon  presence  on  symbol  detection  under 
different  overlap  conditions. 

In  addition  to  detection  performance,  examine  the  ability  of  the  symbols  and  icons  to 
support  the  higher-level  tasks  of  symbol  classification  and  tactical  picture  comprehension. 

Identify  and  experimentally  evaluate  display  options  that  eliminate  symbol  overlap  but 
still  allow  accurate  picture  comprehension  for  use  in  future  tactical  displays.  However, 
given  that  symbol  overlap  is  likely  to  be  an  ongoing  problem  with  current  and  future 
displays  the  effects  of  symbol  overlap  should  continue  to  be  explored  in  future  studies. 
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19.  ABSTRACT 

The  Common  Warfighting  Symbology  Standard  Bravo  version  (MIL-STD-2525B)  is  a  contractual  requirement  for  a  number  of  Australian 
defence  projects.  To  evaluate  how  well  MIL-STD-2525B  supports  rapid  and  accurate  detection  of  symbols  in  clutter  the  current  study 
examined  the  speed  and  accuracy  with  which  23  participants  could  detect  the  presence  of  each  MIL-STD-2525B  air  affiliation  symbol 
when  symbol  overlap  and  icon  presence  were  manipulated.  Detection  efficiency  and  accuracy  were  significantly  lower  for 
MIL-STD-2525B  friend,  assumed  friend,  suspect  and  hostile  symbols  than  for  neutral  and  unknown  symbols.  Symbol  overlap 
significantly  reduced  detection  efficiency  and  accuracy.  Icon  presence  reduced  detection  efficiency  only.  The  results  indicate  that  it  was 
difficult  to  detect  important  air  symbols  in  MIL-STD-2525B.  The  study  also  examined  detection  of  friend,  assumed  friend,  suspect  and 
hostile  symbols  from  the  updated  version  of  the  standard,  MIL-STD-2525C.  The  MIL-STD-2525C  representation  of  suspect  affiliation 
improved  detection  efficiency  for  suspect  and  hostile  symbols  compared  to  MIL-STD-2525B  but  the  representation  of  assumed  friend 
affiliation  produced  poorer  detection  efficiency  for  assumed  friend  and  friend  symbols.  Thus  the  change  to  the  suspect  symbol 
proposed  in  MIL-STD-2525C  appeared  to  be  effective,  but  further  work  is  required  to  modify  the  assumed  friend  symbol  within  the 
scope  of  the  standard  to  improve  the  detection  performance  of  friend  and  assumed  friend  symbols. 
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